Repository navigation
Conversation
|
@DataDog review |
There was a problem hiding this comment.
A profiling signal can interrupt an unguarded native socket sample and wait on the interrupted thread’s unpublished slot. The new null-trace wait can therefore hang an application thread when both samples produce the same stack.
🤖 Bits Code Review · Commit f90a155
CI Test ResultsRun: #44832026667 | Commit:
Status Overview
Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled
|
| Test | Cells | Message |
|---|---|---|
MallocArenaWarningTest.statsEventsDescribeTheArenas__ |
glibc-8-orcl-release-amd64 |
org.opentest4j.AssertionFailedError: expected: <482869248> but was: <477626368> |
These fail the build. Passing on a second run makes a test flaky, not passing. Fix it, or quarantine it against a ticket so the debt is tracked rather than forgotten.
Consider quarantining these — click for ready-made entries
A quarantined test still runs and still reports; its failures just stop
turning CI red. To quarantine one:
- Open a PROF ticket for the test, linking the failing job.
- Append the line below to
ddprof-test/quarantine.txt, replacing
PROF-XXXXXwith the ticket number. - Check the
cellsandreasoncolumns — the proposal only knows what
failed in this run, and a narrowercellsglob keeps the same test
gating everywhere it has not misbehaved.
CI fails once review_by passes, so an entry expires instead of piling up.
# test | ticket | added | review_by | cells | reason
com.datadoghq.profiler.nativemem.MallocArenaWarningTest.statsEventsDescribeTheArenas | PROF-XXXXX | 2026-11-21 | 2027-02-19 | glibc-8-orcl-release-amd64 | org.opentest4j.AssertionFailedError: expected: <482869248> but was: <477626368> (seen in: glibc-8-orcl-release-amd64)
Retried 1 of 101 cells.
Failed Jobs
- glibc-amd64/release / 8-orcl — view logs
Summary: Total: 108 | Passed: 106 | Failed: 1 | Cancelled: 1
Updated: 2026-11-21 01:53:57 UTC
This comment has been minimized.
This comment has been minimized.
|
@DataDog review |
There was a problem hiding this comment.
More details
The socket sampler’s CriticalSection guard covers the same-thread signal re-entry hazard when waiting on unpublished trace slots. Source review found no actionable defect in publication ordering or abandoned-slot retries; runtime and sanitizer behavior remain unverified.
🤖 Bits Code Review · Commit 0811dc8
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
More details
Static review found the publication wait and key revalidation consistent with slot ownership. The socket sampling guard addresses same-thread signal re-entry that could otherwise wait on its own unfinished insertion.
🤖 Bits Code Review · Commit 0811dc8
jbachorik
left a comment
There was a problem hiding this comment.
Sphinx Review
Automated review findings for this pull request are posted as inline comments below.
Reviewed commit: 0811dc8
React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it
What each reaction means
- 👍 Useful: correct, and you're fixing it or will.
- 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
- 😕 Unclear: you can't tell what it means or why it matters.
- 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.
🧞 Sphinx — automated review
Reliability & Chaos Results✅ All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/155385473 |
|
@DataDog review |
There was a problem hiding this comment.
More details
The publication key recheck protects against returning a reclaimed slot’s trace, and the socket sampler’s CriticalSection prevents same-thread signal re-entry into the wait path. The completed static review identified no reportable defect.
🤖 Bits Code Review · Commit 0e7a4a7
What does this PR do?:
Stops
CallTraceHashTable::put()from dropping a trace when two threads insert the same new stack at once.nullptrorPREPARING) is now treated as in progress. The waiter spins until the trace is published, or until the key changes, then probes the slot again. The "preparation failed" drop branch is removed: every abandon path clears the key, so that state never means failure.put()now drops it explicitly before probing. This keeps today's behaviour; without the guard it would wait on an empty slot forever.NativeSocketSamplernow records its sample underCriticalSection, like every otherput()caller. Without it, a profiling signal landing while the socket hook is insideput()could re-enterput()with the same Java-only stack and wait forever on its own thread's claim.Motivation:
PROF-16805. The key CAS and
markPreparing()are two steps. A second thread that probed between them sawkey == hash,trace == nullptrand returnedDROPPED_TRACE_ID. It hits start-up and restart bursts, where many threads first record the same stack (seen as a<dropped>stack inGCGenerationsTeston #663).Additional Notes:
PREPARINGwait already does over the longerstoreCallTrace()window.put()caller now holdsCriticalSection, so a nested signal handler drops its sample instead of waiting on its own thread's slot. The socket path was the one caller without it; this also closes the older self-deadlock in thePREPARINGwindow.PREPARINGbefore the key (a protocol redesign).PREPARINGsentinel;findCallTrace()can create a duplicate ID for an in-progress slot in an older table./code-reviewmedium).How to test the change?:
New deterministic gtests (
CallTraceHashTableWaitTest) use aUNIT_TEST-only wait hook to play the concurrent owner on the waiting thread: claimed andPREPARINGslots, each published and abandoned, reclaim by another stack, spin-limit re-probe, and zero-frame drop. A stress test races 8 threads on the same unseen stacks.NativeSocketSamplerCriticalSectionTestchecks that the socket sample is recorded insideCriticalSectionand dropped when the thread is already in one (Linux-only).For Datadog employees:
dd:platform-security-reviewskill, or file a request via the PSEC review form).bewairealso runs automatically on every PR.